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THE ECONOMIC IMPLICATIONS OF PROGRAMED INSTRUCTION FOR 
WORK TRAINING UNDER CONTROLLED CONDITIONS WERE INVESTIGATED. 
“WORK TRAINING" REQUIRED THE MEMORIZATION OF PROCEDURES 
DURING THE TRAINING PERIOD. ONE EXPERIMENT UTILIZED 20 
UNSKILLED FEMALES WHO COMPLETED A LAYERED ASSEMBLY BY 
REFERRING TO A TYPED LIST OR SLIDES OF THE ASSEMBLY WHICH 
WERE OPERATOR FACED. WITH CRITERIA OF TIME AND ERRORS, THE 
SLIDES WERE MORE EFFECTIVE. A SECOND EXPERIMENT UTILIZED 20 
MALE COLLEGE STUDENTS WHO MADE A LAYERED ASSEMBLY BY 
REFERRING TO A TYPED LIST ON A SLIDE OR SLIDES OF THE 
ASSEMBLY, AGAIN OPERATOR PACED. THE ASSEMBLY SLIDES WERE MORE 
EFFECTIVE THAN THE LIST SLIDES. A THIRD EXPERIMENT UTILIZED 
16 UNSKILLED FEMALES WHO MADE A LAYERED ASSEMBLY BY REFERRING 
TO MULTIPLE SLIDES PER LAYER, SINGLE SLIDES PER LAYER, A 
TYPED LIST IN A TYPEWRITER, OR TAPED AUDIO INSTRUCTIONS. ALL 
WERE OFERATOR PACED. BOTH SLIDE METHODS WERE ABOUT EQUALLY 
EFFECTIVE, AND BOTH WERE MORE EFFECTIVE THAN THE LIST OR 
AUDIO METHODS. A FOURTH EXPERIMENT UTILIZED 10 MALE COLLEGE 
STUDENTS WHO MADE A LAYERED ASSEMBLY BY FOLLOWING A TYPED 
LIST WITH THE FINGERTIP OF THE LEFT HAND, OR USING ONE SLIDE 
PER LAYER. THE SLIDES WERE MORE EFFECTIVE. A FIFTH EXPERIMENT 
UTILIZED 12 MALE COLLEGE STUDENTS WHO MADE A SIMULATED 
ELECTRICAL TERMINAL BOARD BY REFERRING TO A TYPED LIST, 
COLORED SLIDES OF THE ASSEMBLY, OR AUDIO TAPE RECORDING. 
ERRORS INCREASED FROM SLIDES TO LIST TO TAPE. IN ALL 
EXPERIMENTS PICTORIAL PRESENTATION WAS BEST FOR BOTH THE TIME 
AND ERROR CRITERIA. (EM) 
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Interest in programmed learning has increased since Skinner* s article 
"The Science of Learning and the Art of Teaching" in 1954 but most work has 
been oriented toward academic research rather than work training* A decade 
later, Schramm ( 17 ) compiled 160 papers of original research on programmed 
learning* Only 18 dealt with work related tasks and more than 80 percent of 
the 160 experiments used student subjects* We wanted to evaluate the economic 
implications of the new technique for work training under controlled conditions* 

Communication through programmed instruction can be of two types: work 

instruction, in which the operator always has a detailed procedure before 
him, and work training, in which the operator is expected to memorize a par- 
ticular procedure during a training period. Although research in both types 
of communication is currently in progress in the Industrial Engineering De- 
partment at Kansas State University, the remainder of the article will only 
be concerned with work instruction. 

Work done elsewhere 

Communication between the engineer and the worker is typically through 
the visual and/or auditory channels* In general, it has been established 
that the visual channel is better than the auditory channel for transmitting 
information (ll, 15). Cardozo and Leopold (6) had subjects transcribe letters 
and numbers* Visual input results in fewer errors than auditory input. 

Roshal (16) compared the efficiency of presenting information through slides 

t 

and motion picture , for a knot-tying task and found motion pictures better. 
However, he stated "For simple tasks, the most important consideration may 
be the accurate representation of the product of the perceptual motor skill." 
When the subject tied the knot at the same time as the motion picture was 
being shown, Roshal noted that this divided attention seemed to produce a 



conflict* McGuire ( 13 ) felt that motion picture portrayal? of performance 
tend to present information faster than the subject’s input capacity* If 
however, the film speed is slowed down so that all operators can follow it, 
it tends to act as a pacing device for the faster operators. Slides on the 
other hand can be actuated by the individual operator at his own pace* 

Use of the word "visual" is deceiving* Both a typed list of assembly 
instructions and step-by-step colored pictures of an assembly are visual* 
Cross, Noble, and Trunbo (7) in a study on controls found that it was more 
effective to present a picture of what the operator should duplicate (i,e* 
matching behavior) than to give the operator typed directions on how to move 
(i.e. move control left 5" and then down 3"). 

The auditory channel, however, also has its partisans. Goldman and 
Eisenberg (9) found that auditory input of information on an assembly task 

i 

was more advantageous than posting a list of instructions in front of the 
operator. Erlick and Hunt (8) cite as an advantage of using the auditory 
channel that while visual information generally requires either shifting the 
operator’s head orientation or locating the equipment in an often already 
crowded work space, auditory information can be presented by equipment located 
in less critical work areas. Note also that if the operator’s eyes are used 
in the task, this means she reads the information and then performs the task 
(i.e. works sequentially). Giving information to the operator through the 
auditory channel allows her to work "in parallel" as she can obtain infor- 
mation and work at the same time. Thus it appears that both channels have 
their strong and weak points and that any general statement on which is best 
must be applied with caution. 
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The traditional work instruction device is a written or cyped list of 
steps that have been developed by the organizations industrial engineering 
department. This list is posted in front of the operator. Recently more 

sophisticated equipment (l) (2) (3) (4) (o) (19) has been used which have 

« 

tape recorded commentaries in conjunction with color slides. On at least 
one unit (10) the voice is automatically hushed at the end of the instruction 
so the operator can hear background music while carrying out the previous 
instruction. There is very little published evidence of controlled experi- 
ments on work method communication, thus the impetus for the following series 
of investigations. 

EXPERIMENT ONE 

Task . The task, depicted in Fig. 1, used a pegboard with a 4 x 4 matrix of -J-" 
diameter 3” long wooden dowels on 1^" centers. The columns were labeled A-D and 
the rows 1-4. Three types of wooden washers were made: one to fit over one dowel 
(singles), one to fit over two dowels (doubles) and one to fit over three (triples). 
Since components in many industrial assembly tasks must not only be placed 
in the proper position but must also have a specific orientation, the washers 
were colored red on one side of one end of the doubles and triples and white on 
the other side of the other end. The singles were red on one side and white on 
the other. Errors, therefore, could be classified four ways: position (on 
wrong pegs), orientation * (on correct pegs but with ends reversed or part upside 
down), color (on correct pegs but with ends reversed and part upside down), and 
omission (part omitted). Although we have differentiated between color and 
orientation errors. some readers may widh to combine both categories into a single 
category of orientation error. Five singles, four doubles and one triple com- 
posed a layer of non overlapping washers. Twelve different patterns of layers 
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were designed* For experiment one, these twelve layers were grouped into four 
tasks of three layers each. 

The workstation, shown in Fig, 2, was designed for consistency of method 
rather than efficiency. Each operator worked only with the right hand. 
Subjects , Twenty women were hired from the unskilled labor pool of the Kansas 
State Employment Service, Each woman worked for four hours and was paid by 
the hour. Years of schooling ranged from 10 to 17 with an average of 13,0, 
Media , Two methods of work communication were investigated: (l.) The con- 
ventional typed list shown in Fig, 3, (2,) Picture slides taken from the 

operator’s viewing angle. There were three slides for each layer — one for 
the singles, one for the doubles and one for che triple. For clarity the 
"background" components were not shown on the slides. The slides were rear 
projected on the screen of a Hughes Model 202 Video sonic and were changed by 
actuation of a footswitch. Thus both media were operator paced. Note that 
only a small portion of the total information required for the task was 
available on any specific picture wh reas the total task information was 
available on the list. 

Experimental procedure . Each subject used both media and all four tasks. 

In order to balance practice effects ten subjects were randomly assigned to 
the list-picture-picture-list (l-p-p-l) sequence and the other half to the 
p-l-l-p sequence. Each subject completed a total of thirty assemblies (fif- 
teen per task) for the list media and thirty for the picture. The sequence 
of tasks was randomized with the restriction that each task appeared equally 
often in each of the four time periods. 

While the subject worked, the experimenter timed her with a decimal minute 
stop watch. Upon completion of an assembly of three layers, the experimenter 
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pjG. 2. WORKPLACE LAYOUT. 



Fig. 3* Typical instruction list for experiment one. 



TASK I 

1. Get 5 singles 

2. White at Al, C2, D3 

3. Red at B4, C3 

4. Get 4 doubles 

5. White right at Bl, Cl 

6. Red up at B2, B3 

7. Red down at DI, D2 

S. Red right at C4, 04 

9. Get 1 triple 

10 . White down at A2, A3, A4 



11. Get 5 singles 

12. White at A4, Dl, D4 

13. Red at A3, D2 

14. Get 4 doubles 

15. White down Cl, C2 

16. White right B4, C4 

17. White left A2, B2 

18. Red right Al, Bl 

19. Get 1 triple 

20. Red left at B3, C3, D3 



21. 


Get 5 singles 


22. 


White at D3, D4 


23. 


Red at Al, B3, B4 


24. 


Get 4 doubles 


25. 


White up at C3, C4 


26. 


White up at Dl, D2 


27. 


Red down at A3, A4 


28, 


Red left at Bl, Cl 


29. 


Get 1 triple 


30. 


Red right at A2, B2, C2 
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disassembled the unit and noted the type and frequency of errors. This 
infoimation was not given to the subject. Although the inspection time 
varied depending on the number and type of errors, the subject was idle for 
approximately two to three minutes between each assembly. The subjects 
were instructed to work "quickly but accurately". 

Results . Five of the subjects took so long when using the list that they 
would not have been able to complete 15 assetablies of each of the four tasks 
within the four hour experimental session. For example, subject 17 used 
the list for her first task and was still taking over three minutes per 
assembly after 12 assemblies. At that point, the experimenter transferred 
her to the second task and pictures. She completed 15 assemblies of the 
second task and 12 assemblies of the third task using slides. She was 
able to complete 15 assemblies of the fourth task using the list before the 
four hour limit. The data beyond the minimum number completed for any 
one of the four tasks was not used. Thus for subject 17 we used only the 
first 12 cycles of the four tasks. 

There was an average of .060 errors per layer when using the pictures 
ancl .380 errors per layer when using the list; it required an average time 
of .383 minutes per layer for pictures and .815 minutes per layer for the 
list. More detailed information can be found in Tables 1-5. The statistical 
significance of total errors, position errors, orientation errors, color errors, 
omission errors and time was tested with the non-parametric Wilcoxon Matched 
Pairs Signed Ranks test (18). The differences between mddia for each of the 
above criteria were significant (p< .01). 

Tables 1 through 5 indicate that there might be a layer effect. There 
seem to be considerably more errors for the middle and top layers which must 




Table 1. Position errors on experiment one. 







Layer 






Media 


Top 


Middle 


Bottom 


Total 


List— 1st series 


25 


26 


3 


54 


2nd series 


24 


25 


1 


50 


Total/1668 


layers 49 


51 


4 


104 



Pictures— 1st series 


1 


0 


2 


3 


2nd series 


5 


3 


1 


9 


Total/1668 

layers 


6 


3 * 


3 


12 



Total/ 

layer 



.062 



.007 
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Table 2, Orientation errors on experiment one* 



Layer 



Media 


Top 


Middle 


Bottom 


Total 


Total/layer 


List— 1st series 


69 


62 


57 


188 




2nd series 


50 


64 


20 


134 




Total/1668 

layers 


119 


iwwmmmmmm 

126 


77 


322 


.193 



Pictures— 1st series 


9 


8 


13 


30 




2nd series 


8 


5 


6 


19 




Total/1668 

layers 


17 


13 


19 


49 


,029 






Table 3. Color errors on experiment one# 



Layer 



Media 


Top 


Middle 


Bottom 


Total 


Total/layer 


List— 1st series 


24 


32 


16 


72 




2nd series 


14 


27 


11 


52 




Total/1668 

layers 


38 


59 


27 “ 


124 


#074 



Pictures— 1st series 


3 


5 


2 


10 


2nd series 


2 


3 


4 


9 


Total/1668 


5 


8 


6 


19 



layers 



o 



12 



Table 4. Omission errors on experiment dne. 



Layer 



Media 


Top 


Middle 


List — 1st series 


29 


28 


2nd series 


8 


9 


Total/1668 


37 


37 


layers 







Pictures— 1st series 


3 


3 


2nd series 


10 


1 


Total/1668 


13 


4 


layers 







Bottom Total Total/layer 



8 65 

3 20 

11 85 .051 



3 9 

2 _ Jl3 

5 22 



013 
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Table 5. Total assembly time (min.) for experiment one. 



Layer 



Media 


Top 


Middle 


Bottom 


Total/1668 

layers 


Min./ 

layer 


List— 1st series 


268.80 


259.18 


226.18 


754.16 




2nd series 


210.22 


208.27 


187.07 


605.56 




Total 


479.02 


467.47 


413.25 


1359.74 


.813 



Pictures— 1st series 


110.44 


109.44 


110.95 


331.38 




2nd series* 


101.67 


100.55 


104.49 


306.71 




Total 


212.11 


210.54 


215.44 


638.09 


,383 
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be assembled against a background of parts than for the bottom layer which is 
assembled with an "empty" background. This was tested with another non- 
parametric test, Friedman's two-way analysis of variance (18) . Both time 
and position errors had a chi-square value that could have cccured by chance 
less than one time in one hundred so it was concluded that the background 
did affect position errors and time. The probabilities of the chi-square 
occuring by chance for omission errors, orientation errors, and color errors 
were approximately 10 percent, 20 percent and 20 percent respectively, so these 
three are considered to have occured by chance. 

Next the effect of the educational level of the subject was determined. 

The average errors and time per layer were calculated for each subject. The 
difference between the errors for each media for each subject was plotted 
against the subject's educational level. A similiar procedure was followed 
for times. The resulting Spearman rank correlation coefficient (18) was -.494 
for errors and -.721 for times. Both are significant at p < .05. Thus, 
although every woman had more errors and more time for the list than the 
pictures, the difference was inversely related to the educational level and 
those with less formal schooling benefitted most from the pictures. 

After the experiment, the subjects were asked which media they preferred 
and why. Nineteen of the twenty preferred the pictures. One preferred the 
list because "it is more of a challenge". 

Discussion. It was felt that the requirement of "a complex operation with 
a long cycle time" before it was profitable to change from the conventional 
typed list (9) was .incorrect. McCormick ( 12) states that in performing any 
type of work a person engages in essentially three kinds of functions: 

1. Obtaining information 2. Making decisions 3. Acting upon the decisions. 
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It was felt that visual instruction would reduce the time for obtaining in- 
formation and making decisions but probably would not change. the time to carry 
out the decision (eg, soldering time and the time necessary to select the parts 
and assemble the components). Thus the tasks wore designed with a high in- 
formation and decision content and relatively low motor work content. 

The data indicates that pictorial presentation is more effective than 
a typed list. Educational level also is a variable with subjects with less 
education benefitting more from pictures. Adding a component against a com- 
plex background seems to affect some types of errors. 

In order to verify some of these results, a second experiment was run which 
used college students as subjects. Since in experiment one there may have been 
too much information ‘being presented at one time, an attempt was made to equate 
the information content of the two media. Both media were presented by means 
of the Videosonic unit to remove any novelty effects of the machine, 

EXPERIMENT TWO 

Task, Five assemblies of the same type as in experiment one were used, but 
with each assembly consisting of only two layers each instead of the three 
layers per assembly of experiment one. 

Subjects. Twenty male undergraduate and graduate students in industrial 
engineering served without pay. 

Media. In this experiment, the typed instructions from the list were photo- 
graphed and presented on a slide for each layer. The pictures were also on a 
one slide per layer basis instead of the three slides per layer of experiment 
one,. The viewing time for both media was under the control of the subject, 
Jjxpg£iroental procedure. Ten subjects used the list slides and ten used the 
picture slides. The five assemblies were arranged in five different sequences 



0 



16 



to balance sequential effects between tasks# Two subjects used each se- 
quence with each media. One of these two subjects had layer one on the bottom 
and layer two on top while the other had two on the bottom and one on top. 

Thus the five tasks and sequences composed a 5 x 5 Latin square. A ten 
second rest interval was given between each layer with a 30 second inter- 
val between tasks. The subjects were told ” errors and time are equally 
important.” 

Re suits . Table 6 gives a parametric analysis of variance of the time and 
error scores. Errors were not broken into categories as in experiment one. 
There was an average of .32 errors per layer with the list and ,12 errors 
per layer with the pictures. The average time scores were .884 minutes per 
layer with the list and .472 minutes per layer for the pictures. Both of 
these effects were statistically significant; ■ errors at p < .05 and time 
at p < .01. 

The significant display x practice interaction for errors can be 
attributed to the large number of errors committed by the list group on 
the first task. After the first assembly the errors were distributed fairly 
evenly over the remaining tasks. The average error scores for the first 
two layers for the picture were .05 errors per layer and #75 errors per 
layer for the listing. For the last two layers the pictures were .15 errors 
per layer while the listing were #20 errors per layer. 

The subject's time scores decreased with practice to give a significant- 
practice effect. Also practice x top-bottom, and display x practice effects 

were significant for time scores. The significant display x practice reflects 
the fact that the picture times started low and decreased only slightly while 
the list times decreased more rapidly. The average time for the first two 
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Table 6, 

Summary of Analysis of Variance 
for Assembly Times and Error Scores 



Time Scores 
F 



Error Scores 
M.S. F 



Between Subjects 
Display (D) 


19 

1 


8.4914 


46.707** 


2.000 


4.348* 


Blocks (B) 


4 


.2446 


1.345 


.930 


2.022 


D x B 


4 


.2341 


1.287 


.450 


.978 


Error 


10 


1.8175 








Within Subjects 
Practice (P) 


180 

4 


.3555 


44.438** 


.718 


2.124 


Target (T) 


4 


.0185 


2.312 


.080 


.237 


Order (TB) 


1 


.0103 


1.287 


.020 


.059 


P x TB 


4 


.0688 


8.600** 


.258 


.763 


T x TB 


4 


.0103 


1.287 


.045 


.133 



D x P 
D x T 
DxTB 
3 xTB 
D x TB x T 
DxBxTB 
D x IB x P 
Error 



4 

4 

1 

4 

4 

4 

4 

138 



.1708 

.0166 

.0134 

.0063 

.0012 

.0119 

.0071 

.0080 



21 . 350 ** 
2.750* 
1.675 
.788 
.150 
1.487 
.887 



.913 

.275 

.021 

1 

.620 
f 020 
.270 
.283 
.338 . 



2.701* 

.814 

.059 

1.834 

.059 

.799 

.837 



*£ < * 05 

< .O'i 
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layers for the picture group were ’.513 minutes and for the listing were 1.154 
minutes, the average time for the last two layers were ,439 minutes for the 
picture and ,775 minutes for the listing. The significant practice x top- 
bottom interaction reflects thg failure for all bottom layers to take longer 
than the associated top layers. That is, initially the bottom layer took 
longer than the top layer, but after the initial learning period the top 
was taking longer than the bottom. The reason for the top taking longer 
than the bottom was probably due to the confusing background while building 
the top layer which was not present while building the bottom layer. 

Discussion . From the time scores we see that the listing Started out con- 
siderably higher than the pictorial and remained so through the five assem- 
blies; however, there were indications that the difference between the 
media may converge. AT. so the data indicated that the error scores will 
converge much sooner than the time scores. 

The results of this experiment show that even with the information 
content of the media equated and with removal of any novelty effects the 
pictorial display remains superior. To make a more direct comparison, ex- 
periment three was designed to compare the effect of changing the information 
content per presentation. The effect of the use of the audio channel of 
presentation was also investigated. 

EXPERIMENT THREE 

Task . The four assemblies of experiment one were repeated. 

Subjec ts. Sixteen women were hired from the unskilled labor pool of the 
Kansas State Employment Service. Each woman worked approximately four hours 
and was paid by the hour. Average years of schooling completed was 12.75. 
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Media, Four different media were used: (l,) Three picture slides per layer 

of experiment one, (2,) One picture slide per layer similiar to those in 
experiment two, (3.) The list of experiment one. However, the list was 
placed in a typewriter which permitted the subject to keep track of her place 
as she turned the roller, (4,) Audio instructions. These were given through 
the tape recorder portion of the Videosonic with the letters given as the 
words "able”, "baker", "Charlie", and "dog". The instructions were the same 
as those on the list. After the instruction to get the part, there was a 
pause of 1,8 seconds before the position instructions were given, . There was 
another pause of 1,2 seconds at the end of the position instruction before 
the new get instruction. If desired, the tape could be stopped by depressiing 
a footswitch so this media as well as the others was operator paced. Hie 
running times for the tapes if they were not stopped were 2:37, 2:35, 2:35, 
and 2;45 minutes for tasks 1, 2, 3, and 4, 

Experimental procedure . Each subject used each of the media for 15 consecutive 
assemblies. The four media and the four tasks composed a 4 x 4 Graeco-Latin 
square of subjects by sequence. Subjects 5-8 and 9-12 used different squares 
but subjects 13-16 repeated the same square as subjects 1-4. The subjects 
were instructed to work "quickly but accurately". 

Results . Error categories are shown in Tables 7 to 10 with an average of .290 
errors per layer when using the list, .592 when using audio instruction, .071 
when' using one picture per layer, and .103 when using three pictures per layer. 
Using a two tail Wilcoxon test for comparing these errors, the difference 
between the one picture per layer and three pictures per layer was significant 
at p < .05 and the differences between three pictures and list and between 
list and audio were significant at p < .01, 
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Table 7. Position errors in experiment three. 



Layer 



Media 


Top 


Middle 


Bottom 


Total Total/720 layers 


One picture/ 
layer 


4 


3 


3 


10 


.014 


Three pictures/ 
layer 


7 


4 


7 


18 


.025 


List 


47 


19 


8 


74 


.103 


Audio 


87 


82 


42 


211 


.293 
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Table 8. Orientation errors in experiment three. 







Layer 






Media 


Top 


Middle 


Bottom 


Total 


Total /720 layers 


One Picture/ 
layer 


10 


4 


12 


28 


.039 


Three Pictures/ 
layer 


12 


10 


22 


44 




.061 


List 


27 


30 


26 


83 


.115 


Audio 


34 


37 


19 


90 


.125 
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Table 9. Color errors on experiment three. 



Layer 



Media 


Top 


Middle 


Bottom 


Total 


Total/720 layers 


One Picture/ 


3 


k 


0 


7 




.010 


layer 














Three Pictures/ 


h 


5 


2 


11 




.015 


layer 


List 


20 


15 


15 


50 




• 

o 

ON 

VO 


Audio 


>3 


3** 


9 


86 




.119 




Table 10 



Omission errors on experiment three. 



Media 


Top 


Layer 

Middle 


List 


0 


2 


One Picture/ 


layer 


3 


1 


Three Pictures/ 


layer 


5 


3 


Audio 


12 


18 



Bottom 


Total 


Total/720 layers 


0 


2 


.003 


2 


6 


oo 

o 

o 

• 


1 


9 


.012 


10 


1*0 


.055 
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For position errors alone (Table 7) the difference between the one pic- 
ture per layer and three pictures per layer was not significant, but both the 
differences between three pictures per layer and the list, and between the 
list and audio we re significant at p < ,01. Therefore, the difference between 
one picture per layer and the list is also significant. 

For orientation errors alone (Table 8) the difference between the one 
picture per layer and three pictures per layer was not significant, and neither 
was the difference between three pictures per layer and the list. However, 
the difference between three pictures per layer and audio was significant 
(p < .05) as was the difference between the one picture per layer a. 1 the 
list (p < .01). 

For color errors alone (Table 10) the differences between the list, one 
picture per layer, and three pictures per layer were not significant, but the 
difference between three pictures per layer and audio was significant (p < .05). 

The possible layer effect (ie. the effect of a confusing background) in-' 
dicated in Tables 7-10 v/as tested with Friedmans tv/o way analysis of variance 
(18). For position errors (Table 7), the layer effect was significant (p < ,05) 
only for the audio and list media. For orientation errors (Table 8), the layer 
effect was not significant for any of the media. For color errors (Table 9), 
the layer effect was significant only for the audio media. For omission errors, 
(Table 10), the layer effect was not significant for any of the media. 

The time per layer for the three picture per layer group v/as .47 minutes, 
for one picture per layer was .50 minutes, for the audio was 1,01 minutes, 
and for the list was 1.08 minutes. The difference between the .47 and .50, 
and 1.01 and 1.08 were not significant but the difference between the .50 
and 1.01 was significant (p < ,0}.), 



o 
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The errors made by each subject while using the one picture per layer 
was subtracted from the errors she made while using the list* The difference 
was correlated in the expected direction with education but the correlation 
(r = -.240) was not statistically significant* The differences between the 
list and three pictures per layer had a correlation of -.200, the differences 
between the audio to the one picture per layer had a correlation of -.378, 
and the differences between the audio and three pictures per layer had a 
correlation of -.354* These three correlations are also in the expected direction 
but not statistically significant. The correlation between differences in 
time between the list and three pictures was significant (-.515) but the dif- 
ferences in time between audio and three pictures per layer was not significant. 

Each subject was asked to indicate v/hich media was "best" and which "worst". 
Best was scored at "+l u and worst as "-1". When the scores for the media 
were totaled for the sixteen subjects, the three pictures per layer haid a 
score of 8, one picture per layer had 4, audio was 0, and the list was -12, 
Discussion . The results of experiment three show some small differences be- 
tween the one and three pictures per layer groups. However, this difference 
is probably small enough to neglect in an actual work situation. Audio pre- 
sentation seems to be a very poor way of giving work instructions regardless 
of whether time or error scores are used as a criterion. Listing, how- 
ever, proved to be only slightly better than the audio. A possible reason 
for the poor performance when using audio is referability. When the subject 
used the tape she either understood it the first time or it was lost and no 
reference was possible. With the pictures the subject could refer back with- 
in a specific instruction although not to previous instructions while with 
the list she could refer back as far as she wished. 
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Position errors were shown to be significantly affected by a confusing 
background in both experiment one and three when a list is used although pre- 
senting information through pictures did not bring out this problem. In the 
following experiment we wished to investigate both the layer effect and refer- 
ability in more detail. 

EXPERIMENT FOUR 

Task. Two six layer assemblies were formed by combining task one and two, 
and task three and four of experiment one. 

Subjects . Ten male undergraduate students were paid by the hour. Average 
years of education completed was 14,5. All subjects were right handed. 

Media . Two media were used: (l.) The typed list of experiment one. The 

subjects, however, were told to keep their place on the list with a finger 
of their left hand. (2.) The one picture slide per layer of experiment 
three. Both media were operator paced. 

Experimental procedure . Each subject used each media for eight assemblies. 

The experiment was counterbalanced by having subjects 1-5 assemble four 
units for each condition using the sequence: List-Task 1, Picture-Task 2, 

Picture-Task 2, and List-Task 1. Subjects 6-10 followed the sequence 
P-1, L-2, L-2, P-1. They were instructed to work "quickly but accurately". 
Results . The error scores are given in Table 11. Using a Wilcoxon test, the 
position and omission errors were not statistically different across media, 
but the difference in color errors was significant (p < .05) while orientation 
errors and total errors were significantly different (p < .01). The average 
time of .72 minut.es per layer for the list was significantly (p < .01) dif- 
ferent from the .41 minutes for the picture. The effect of background (ie. 
layer effect) was not significant for either media. 
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Discussion , Refer to Table 12 for a comparison of the error rates and time 
scores for the different media for the previous experiments. Average errors 
per layer ranged from ,060 to .133 for pictures while the list varied from 
,222 to .380 per layer and rhe audio had .592 errors per layer. Average 
times per layer ranged from .38 to .50 for pictures while the list varied 
from .72 to 1.08 and the audio had 1.01 minutes per layer. Thus, in general, 
pictures would be preferred over the list which would be preferred over the 
audio. 

A closer look at the types of errors in interesting. The advantage of 
a place keeping device for the list seems to result in a decrease in omission 
errors and orientation errors while color errors decrease only slightly and 
position errors even seem to increase. Audio orientation errors are approxi- 
mately three times greater than one picture orientation errors,, but omissson 
errors are seven times greater, color errors are twelve times greater and 
position errors are fourteen times greater. 

Another example of how changes in media affect errors was observed in 
the color errors for experiment four. There were five singles per layer, 

48 layers per subject per media, and ten subjects of a total of 2400 oppor- 
tunities for errors on singles. The four doubles and one triple per layer 
also afforded 2400 opportunities for error. The subjects made 15 color errors 
on singles when using the list and 6 (40 percent of the list) when using the 
pictures. However, they reduced the color errors on doubles and triples 
from 13 with the list to only one (8 percent of the list) when using the 

pictures. Thus even different types of parts are differentially affected 
by changes in media. 

Experiment five was designed to further analyze three of the media 




Table 12. Comparison of the average error and 
time scores per layer for experiments one through four. 



Experiment 


Types of Errors 

Media Position Orientation Omission 


Color 


Average 

errors/ 

laver 


Average 

min./ 

laver 


One 


List 


.062 


.193 


.051 


.074 


.380 


.813 




Three Pictures 


.007 


.029 


.013 


.011 


.060 


.380 


Two 


List on Slide 


...... 


...... 


MMM 


MIWOT 


.320 


.880 




One Picture 




— - 


M*NM 




.120 


.470 


Three 


List (typewriter) 


.103 


.115 


.003 


.069 


.290 


1.080 




Three Pictures 


.025 


.061 


.012 


.015 


.133 


.470 




One Picture 


.014 


.039 


.008 


.010 


.071 


.500 




Audio 


.293 


.125 


.055 


.119 


.592 


1.010 


Four 


List (finger) 


• 066 


.094 


.004 


.058 


.222 


.720 




One Picture 


.013 


.033 


.006 


.015 


.067 


.410 
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previously studied (list* picture, and audio) but with a different type of 
assembly# 

EXPERIMENT FIVE 

Goldman and Eisenborg (9) showed that for a task involving selecting, 
bending leads, and installing 15 resistors in a terminal board, recorded 
auditory work instructions were significantly better (standard time was 
reached in less time) than use of a printed list. Several factors were in- 
fluential in the decision to replicate their experiment; the results of 
experiment three, their use of the sole criterion of number of cycles to reach 
standard rather than average time per cycle or errors per cycle, and the 
opportunity to compare our results vs. results obtained by another experi- 
menter operating under different conditions# 

Their experiment was modified in the following ways: 1. Another 

media, colored pictures presented on slides, was added to their two media. 

2, Each subject served in each condition rather than their independent groups 
type of experiment, 3. The number and type of errors was recorded. 4. College 
freshmen were used instead of industrial workers. 5. The experimental task 
was modified slightly. 

Task. A maple board with 63, 5/32 inch diameter holes arranged in the pattern 
shown in Figure 4 was painted flat black. The 1/8 inch high numbers were made 
from green pressure- sensitive plastic tape. One hundred standard IBM per- 
manent-type plugged wires were substituted for the resistors# Each wire was 2-J- 
inches long and had a 7/8 inch long plug on each end. These gray wires were hand- 
painted with three, bright enamel stripes to correspond to the resistor colors# 
Either end of the wire could be placed in either hole so there were no 
orientation errors. Groups of five wires of each of the ten color codes were 




Pictorial instructions for experiment five. 
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placed in an unpaiftted pine parts board Having 3/8 inch holes one inch apart 
in a 10 x 10 matrix. Thus there were 100 h§les for the 50 wires. TR?§e 
highly similar tasks were used, Task A, Task B, and Task C. Task B u§ed the 
same sequence of color codes as Task A but the hole V cations were mirror 
images of those in Task A, Task C was Task A in r' verse order* 

Subjects . Twelve male freshmen who were ndt cole.- blind were paid fey th® hour. 
Media . Three media were used. Figure 5 gives he typed instruction list 
Ta sk A, Subjects could use their non-preferred hand (they could use oniy their 
preferred hand to assemble) to keep their place on the list but this was not 
suggested to them. 

The instructions from the typed list were read verbatim onto a tape as 
Quickly as possible while still remaining intelligible. The average time 
per instruction m§ three seconds with a 1.2 second pause between instructions. 
This speed, which was satisfactory for experienced operators, was too fast for 
beginners, so they used their non-pief erred hand to manipulate the on-off switch 
of the tape recorder. They were not permitted to reverse the tape to repeat 
a missed instruction. 

An example of the colored slides taken for each instruction is given in 
Figure 4. Each picture showed only the wire to be inserted with no "background" 
components except the board itself. A red-tipped yellow pencil was used to 
indicate the location of the holes on the board. An oversized replica of 
the wire color code and the pertinent hole numbers appeared in one corner of 
the picture. The slide was projected on a screen directly in front of the 
operator by a Kodak Carousel projector. The subject controlled the indexing 
of the slides with a remote hand switch and was permitted to back up to the 
previous slide if he wished. 
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Fig. 5. Typical instruction list for experiment five. 



TASK A 



Green-Brown-Red 


to 


20-26 


Orange-White-Brown 


to 


43-50 


Red-White-Orange 


to 


30-34 


Brown-Black-Orange 


to 


7-8 


B? ue-Gray-Orange 


to 


22-24 


Yellow-Violet-Orange 


to 


13-17 


Br own-Gr e en-Red 


to 


44-45 


Red-White-Orange 


to 


55-60 


Y ellow-Violet-Orange 


to 


18-23 


Green-Blue-Red 


to 


3-9 


Orange-White-Orange 


to 


42-47 


Orange-White-Orange 


to 


29-33 


Red-White-Brown 


to 


11-12 


Blue-Gray-Orange 


to 


51-58 


Brown-Black-Orange 


to 


2-5 



L /ou-f'jinontal procedure, Hvery subject always worked through 15 assemblies of 
T^k A, anc * C in that order. The three media (picture, audio, and list) 
have six possible sequences. Two subjects used each of these sequences in 
an experimental design which required each media to occur fi^st, second, and 
third equally often for each task. 



Each subject was given a demonstration of how to complete one assembly of 
Task 1. In addition, brief demonstrations of how to use the slide projector 
and tape recorder were given before starting the task requiring those media. 

The subjects were given no rest between assemblies and five minutes between 
tasks. As the subject worked, the experimenter quickly replaced wires used in 
the previous assembly on the parts board used in the previous assembly and then 
recorded the types of errors as they occurred. At the end of an assembly he re- 
corded assembly time, removed wires from the assembled board, and gave a new parts 
board to the subject. Meanwhile, the subject prepared the media for the next cycle. 
The subject was given no feedback on time or errors, although they requested it. 
ResujMts. A Dixon criterion (14) was used to test for outliers; subject seven 
had a long average assembly time and subject ten had considerable errors but 
neither subject could be eliminated. 

Errors are shown in Table 13. The position errors were significantly 
(p < .01) less for the picture when comparing them with the tape, but the 
list was nox significantly different from the picture or tape. Omission errors 
were significantly (p < ,01) less for the picture than for both the tape or the 
)ist, but the difference between the tape and the list was non-significant. 

None of the differences in color errors between the media were significant. 

When all errors were combined the pictures were significantly (p < .01) better 
than the tape, and the list was significantly better (p < .05) than the tape, 
but the difference between the picture and the list was non-significant. 



Table 13. Errors for experiment five. 



Type of Error 



Media 


Position 


Omission 


Color 


Total 


List 


19 


21 


35 


75 


Picture 


k 


1 


1*3 


1*8 


Tape 


2k 


59 


62 


11*5 



o 
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The average time for 15 assemblies when using the tape was 24*85 min- 
utes, for pictures was 29.19 minutes and for the list was 31.16. Using 
Wlicoxon tests with two tails, the tape v/as significantly better than both 
the picture and the list but the picture was not significantly different than 
the list. It is quite likely that the 59 omitted wires when using the tape 
(approximately 2 % of the 2500 assembled when using the tape) contributed 
to the lower time for the tape. 

DISCUSSION 

The primary objective of the paper v/as to investigate the differences 
between forms of work instruction. The most obvious result of these experi- 
ments was that pictorial presentation is the best using either a time or an 
error criterion. 

The types of errors committed with the different media are also interesting. 
Experiments three, four, and five demonstrated that media do not affect all 
types of errors equally. For example, in experiment five there were almost 
five times as many position errors when using the list as when using the pic- 
ture. This did not mean, however, that there would be five times as many color 
errors. There actually were more color errors for the picture than with the 
list, therefore, making it extremely hard to generalize. 

Visualization also seems to be an important consideration. If a subject 
either made a mistake or suspects he made a mistake when using the picture, 
he merely looked at the picture and matched what he saw in the picture with 
what he haid assembled. When using the list, he had to decode the written words 
and form a mental picture and compare this mental picture with his actual 
assembly. When using audio instructions, visualization was not merely necessary 
as with the list but also required a good memory. Experiment three and four when con 
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trasted with experiment one, demonstrated that keeping your place on a list 
(either with your finger or the typewriter carriage) decreased orientation 
and omission errors although it did not decrease color or position errors. 

When using pictures, there seems to be an information content per message 
effect. That is, it may be that too much information is being presented on 

N. 

a picture. This was noticeable when the same information was presented in 
experiment three on either one slide or three separate slides. More slides 
for a given amount of information means the subject has less searching and 
interpreting to do. Fewer slides for a given amount of information means less 
indexing time between slides, possibly better referability since more infor- 
mation is available for reference, and of course fewer slides. 

Audio presentation of material seems to be the poorest method from both 
a time and error viewpoint. One problem is the lack of referability. Another 
problem is that the audio seems to act as a pacing device. The subject improves 
only up to the pace of the tape and then maintains the pace of the tape. A 
slow tape sometimes frustrates the subject because he feels he is being held 
back. On the other hand, it was noticed (especially in experiment five) that 
the tape performs a feedback function in that it gives some knowledge of re« 
suits to the subject as he works. In experiment five the subjects counted 
the number of times they had to stop the tape or counted the number of wires 
positioned before stopping the tape for the first time. Since not all char- 
acteristics of the tape are bad, it may be very useful when used in con- 
junction with other media. 

The longest .time any individual operator was studied was four hours in 
experiment one and even in this experiment only two hours were spent in any 
one media. Certainly longer term studies need to be made to see at what num- 
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ber of cycles the media become equivalent. Since synchronization of a tape 
and slides is expensive, investigation of other methods of presenting com- 
plex material as well as just tape and slides are in progress. Although 
these experiments have demonstrated differences in work instruction media in 
a series of controlled university experiments, it is felt that much work 
remains to be done* 
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